Table of Contents¶
- Objective
- Data Preparation
- One-Hot Encoding
- Evaluation Functions
- Neural Network Models
- Convolutional Neural Network
- Convolutional Neural Network with OHE Industry Sector Input
- Fully Connected Neural Network
- Recurrent Neural Network with LSTM
- Recurrent Neural Network with LSTM and OHE Industry Sector Input
- Recurrent Neural Network with GRU
- Recurrent Neural Network with GRU and OHE Industry Sector Input
- Convolutional Neural Network
- Discussion and Outlook
Objective¶
- Predict seemingly random Stock Market Changes
- Try to find Underlying Patterns
- Evaluate Time Series as Prediction Factor
- Use known effect of Industry Correlation for Prediction
Data and Preliminaries¶
- adjusted close values
- N = 20 stocks
$t_{min} = 2007 \, ,\quad t_{max} = 2017$
Input Data
- Adjusted Close Values
- Return Values
- Implied Volatility Values
In [6]:
begin = "2007-01-01"
finish = "2017-01-01"
N=20
#data_loader(begin, finish, 'returns.csv')
returns = get_returns('returns.csv', N)
delete_nans('returns.csv', 'returns.csv')
data_head('returns.csv', N)
In [7]:
plot_example_returns1('returns.csv', N)
In [8]:
def get_data_subsets(df, dur, variant, onehot):
"""
variant options: month, vola, next_day
"""
branches_ = pd.read_csv('snp500info.csv', index_col=0)
branches = branches_.drop(columns = ['Security', 'GICS Sub Industry', 'start at yahoo'])
companies = list(df)
data_onehot = branches.loc[branches.index.isin(companies)].iloc[:,0]
# integer encode
label_encoder = LabelEncoder()
integer_encoded = label_encoder.fit_transform(data_onehot)
# binary encode
onehot_encoder = OneHotEncoder(sparse=False)
integer_encoded = integer_encoded.reshape(len(integer_encoded), 1)
onehot_encoded = onehot_encoder.fit_transform(integer_encoded).T
df = np.array(df)
month = 30
leng = df.shape[0]
x=[]
y=[]
for i in range(leng):
if dur+i>=leng:
break
if onehot:
x_inp = np.concatenate((onehot_encoded, df[i:dur+i,:]))
else:
x_inp = df[i:dur+i,:]
if variant == 'vola':
x.append(np.abs(x_inp))
y.append(np.abs(df[dur+i,:]))
else:
x.append(x_inp)
if variant == 'month':
y.append(np.mean(df[dur+i:dur+i+month,:], axis = 0))
elif variant == 'next_day':
y.append(df[dur+i,:])
else:
raise NameError("Use correct fun_label")
return np.array(x),np.array(y)
One Hot Encoding of Branch Information¶
In [9]:
pd.read_csv('snp500info.csv', index_col=0).head()
branches_ = pd.read_csv('snp500info.csv', index_col=0)
branches = branches_.drop(columns = ['Security', 'GICS Sub Industry', 'start at yahoo'])
branches.head()
Out[9]:
In [10]:
data = branches.iloc[:,0]
# integer encode
label_encoder = LabelEncoder()
integer_encoded = label_encoder.fit_transform(data)
print(integer_encoded)
In [11]:
# binary encode
onehot_encoder = OneHotEncoder(sparse=False)
integer_encoded = integer_encoded.reshape(len(integer_encoded), 1)
onehot_encoded = onehot_encoder.fit_transform(integer_encoded)
df = pd.DataFrame(onehot_encoded)
df.head()
Out[11]:
Evaluation Functions¶
Evaluation Types:
- Sign test
- Predicition vs Result
- Simultaneous Plot over Time
- Top 5 Portfolio Test
Sign Test¶
- Compares Signs of Model Prediction and Data
- Absolute Test of Prediction Quality
In [12]:
def directional_goodness(model,x_test,y_test):
"""
directional goodnes gives the factor of correctly predicted signs of first order derivative of returns to false ones
oder auch:
gibt die Anzahl der Beobachtungen an, deren Vorhersage das richtige Vorzeichen hatte (Kurs steigt, Kurs fällt)
"""
try:
y_pred = model.predict(x_test)
except AttributeError:
y_pred = model(x_test)
count = 0
#print(y_new.shape)
for i in range(len(y_pred)):
for j in range(len(y_pred[i])):
p= y_pred[i,j] > 0
n = y_test[i,j] > 0
if n==p:
count +=1
print('percentage of correctly predicted directions of returns: ' + str(count/len(y_pred)/len(y_pred[0])))
Top 5 Portfolio Test¶
- Select 5 stocks with highest expected portfolio
- Calculate Mean Return of top 5 stocks
- Compare with Mean Return of all Stocks
- Relative Test of prediction quality
In [13]:
def largest(array, amount):
return array.argsort()[:,-amount:]
def compare_top(model, x_test, y_test):
try:
y_pred = model.predict(x_test)
except AttributeError:
y_pred = model(x_test)
amount = 5
mean_all = np.mean(y_test)
top_index = largest(y_pred,amount)
mean_mach = np.mean(y_test[top_index])
ret = mean_mach-mean_all
print('Difference between mean return and return of calculated top 5: ' + str(ret))
return ret
Prediction vs Result¶
- Scatter Plot Prediction/Result
- Visualize the Data
In [14]:
def internal_goodness(model, x_new, y_new):
print('compare true to predicted values of internal validation on learned dataset: ')
y_pred = model.predict(x_new)
#model.predict nimmt x-werte und gibt die predicteten y-werte zurück
plt.scatter(y_new, y_pred)
Simultaneous Plot over Time¶
- Plot both expected Returns and Actual Returns over time
- Visualize the Data
In [15]:
def external_goodness(model, x_new, y_new):
print('compare true to predicted values of external validation set: ')
y_pred = model.predict(x_new)
color = ["r", "b", "g"]
for i in range(3):
plt.plot(y_new[:20,i],c=color[i], label = 'true')
plt.plot(y_pred[:20,i], c=color[i], linestyle='--', label = 'pred')
plt.legend()
Models¶
- Moving Average
- 1D CNN
- RNN
- LSTM
- GRU
- Fully Connected Network
In [17]:
# Mean über alle returns, mean über ausgewählte Daten
print(np.mean(returns))
print("overall mean = " + str(np.mean(np.mean(returns))))
In [18]:
def mov_avg(x_train):
print(np.array(x_train).shape)
return np.sum(x_train, axis = 1)/x_train.shape[1]
In [19]:
# directional_goodness(mov_avg, x_test, y_test)
# compare_top(mov_avg, x_test, y_test)
# plots(mov_avg, x_test=x_test, x_train=x_train, y_test=y_test, y_train=y_train)
In [20]:
def cnn(x_train, y_train, x_test, y_test, inp_shape, epochs):
verbose, batch_size = 0, 50
epochs = epochs
"""
verbose: 0-kein output, 1-ladebalken, 2-epochenzahlen printen
batch_size: Nicht definieren (https://stackoverflow.com/questions/44747343/keras-input-explanation-input-shape-units-batch-size-dim-etc)
epochs: Anzahl Iterationen durch das Trainingsset
"""
N = inp_shape[1]
#init
model = models.Sequential()
model.add(layers.Conv1D(filters=64, kernel_size=3, activation='relu',
input_shape=inp_shape))
model.add(layers.Conv1D(filters=64, kernel_size=3, activation='relu'))
#Conv1D => 1D Convolution (Schaut nach Mustern)
#mit filters = Anzahl Weight Functions, kernel_size = Anzahl simultan betrachteter Felder,
#relu = 0 für value<0 sonst linear
model.add(layers.Dropout(0.3))
#Dropout sets randomly chosen values to 0 to prevent overfitting
model.add(layers.MaxPooling1D(pool_size=2))
#MaxPooling halbiert array Größe und nimmt größte Werte der Feature-Gewichtungen
model.add(layers.Flatten())
#Flatten reduziert dimensionen eines arrays auf niedrigst mögliche Dimension (1xdim) (überschreibt Nullen))
model.add(layers.Dense(100, activation='relu'))
model.add(layers.Dense(N, activation='linear'))
#Klassisches NN hinter Convolutional Layer geschaltet, lernt also im Feature Raum, durch Convolutional Net vorgebenen
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mse', 'mae'])
#mean_squared_error (mse) and mean_absolute_error (mae)
#fit network and write in history
history = model.fit(x_train,y_train, epochs=epochs, #batch_size=batch_size,
verbose=verbose)
#evaluate model
#Print error values for classification of goodness
mse,mse2,mae = model.evaluate(x_test,y_test, batch_size=batch_size, verbose=verbose)
print(mse)
print(mse2)
print(mae)
return model,history
In [21]:
def main_cnn(input_df, dur, limit, variant, epochs):
"""
input_df die Input Matrix (duh)
dur Zeitfenster
limit Grenze für Train Test Split
variant prediction of next day, next month average, volatility
"""
N = input_df.shape[1]
D = input_df.shape[0]
train = input_df.iloc[:limit,:]
test = input_df.iloc[limit-dur:,:]
x_train,y_train = get_data_subsets(train, dur, variant="next_day", onehot=0)
x_test,y_test = get_data_subsets(test, dur, variant="next_day", onehot=0)
inp_shape = (dur,N)
x_train = x_train[:,x_train.shape[1]-dur:,:]
x_test = x_test[:,x_test.shape[1]-dur:,:]
"""für Dimensions-tests"""
print('x-train shape: ' + str(x_train.shape))
print('y-train shape: ' + str(y_train.shape))
print('x-test shape: ' + str(x_test.shape))
print('y-test shape: ' + str(y_test.shape))
print('test shape: ' + str(test.shape))
model,history = cnn(x_train, y_train, x_test, y_test, inp_shape, epochs)
return model,history,x_train,y_train,x_test,y_test
Next Day¶
In [22]:
model_cnn,history_cnn,x_train,y_train,x_test,y_test = main_cnn(returns, 100, 2000, variant='next_day', epochs=200)
print(model_cnn.summary())
In [23]:
directional_goodness(model_cnn, x_test, y_test)
compare_top(model_cnn, x_test, y_test)
plots(model_cnn, history_cnn, x_test=x_test, x_train=x_train, y_test=y_test, y_train=y_train)
Next Month Average¶
In [24]:
model_cnn_m,history_cnn_m,x_train_m,y_train_m,x_test_m,y_test_m = main_cnn(returns, 100, 2000, variant='month', epochs=200)
print(model_cnn_m.summary())
In [25]:
directional_goodness(model_cnn_m, x_test_m, y_test_m)
compare_top(model_cnn_m, x_test_m, y_test_m)
plots(model_cnn_m, history_cnn_m, x_test=x_test_m, x_train=x_train_m, y_test=y_test_m, y_train=y_train_m)
Implied Volatility¶
In [26]:
model_cnn_v,history_cnn_v,x_train_v,y_train_v,x_test_v,y_test_v = main_cnn(returns, 100, 2000, variant='vola', epochs=200)
print(model_cnn_v.summary())
In [27]:
directional_goodness(model_cnn_v, x_test_v, y_test_v)
compare_top(model_cnn_v, x_test_v, y_test_v)
plots(model_cnn,history_cnn_v, x_test=x_test_v, x_train=x_train_v, y_test=y_test_v, y_train=y_train_v)
Expectations:
- CNN learns patterns in stock market changes, if there are any recognisable patterns
- Patterns lead to better predictions
Results:
- No Patterns were recognised, or influence of recognised patterns was diminishing/not contributing to better predictions
- Prediction of Next Month Average were overestimating low values und underestimating high values
- Prediction in Volatility Space were uoverestimating low values and underestimating high values
In [28]:
def cnn_2inputs(xtrain,ytrain,xtest,ytest,inp_shape1,inp_shape2,epochs):
verbose, batch_size = 0, 50
epochs = epochs
"""
verbose: 0-kein output, 1-ladebalken, 2-epochenzahlen printen
batch_size: Nicht definieren (https://stackoverflow.com/questions/44747343/keras-input-explanation-input-shape-units-batch-size-dim-etc)
epochs: Anzahl Iterationen durch das Trainingsset
"""
i1 = layers.Input(inp_shape1)
i2 = layers.Input(inp_shape2)
N = inp_shape1[1]
#init
model1 = layers.Conv1D(filters=64, kernel_size=3, activation='relu')(i1)
model1 = layers.Conv1D(filters=64, kernel_size=3, activation='relu')(model1)
#Conv1D => 1D Convolution (Schaut nach Mustern)
#mit filters = Anzahl Weight Functions, kernel_size = Anzahl simultan betrachteter Felder,
#relu = 0 für value<0 sonst linear
model1 = layers.Dropout(0.5)(model1)
#Dropout sets randomly chosen values to 0 to prevent overfitting
model1 = layers.MaxPooling1D(pool_size=2)(model1)
#MaxPooling halbiert array Größe und nimmt größte Werte der Feature-Gewichtungen
model1 = layers.Flatten()(model1)
#Flatten reduziert dimensionen eines arrays auf niedrigst mögliche Dimension (1xdim) (überschreibt Nullen))
model1 = models.Model(inputs = i1, outputs = model1)
### second network:
model2 = layers.Flatten()(i2)
model2 = layers.Dense(100, activation="relu")(model2)
#model2 = layers.Dense(100, activation="relu")(model2)
model2 = models.Model(inputs = i2, outputs = model2)
combined = layers.concatenate([model1.output, model2.output])
model = layers.Dense(100, activation='relu')(combined)
model = layers.Dense(N, activation='linear')(model)
model = models.Model(inputs = [model1.input, model2.input], outputs = model)
#Klassisches NN hinter Convolutional Layer geschaltet, lernt also im Feature Raum, durch Convolutional Net vorgebenen
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mse', 'mae'])
#mean_squared_error (mse) and mean_absolute_error (mae)
#fit network
history = model.fit(xtrain,ytrain, epochs=epochs, #batch_size=batch_size,
verbose=verbose)
#evaluate model
#Print error values for classification of goodness
mse,mse2,mae = model.evaluate(xtest,ytest, batch_size=batch_size, verbose=verbose)
print(mse)
print(mse2)
print(mae)
return model,history
In [29]:
def main_cnn_2inputs(input_df, dur, limit, variant, epochs):
"""
input_df die Input Matrix (duh)
dur Zeitfenster
limit Grenze für Train Test Split
variant prediction of next day, next month average, volatility
"""
N = input_df.shape[1]
D = input_df.shape[0]
train = input_df.iloc[:limit,:]
test = input_df.iloc[limit-dur:,:]
x_train,y_train = get_data_subsets(train, dur, variant, onehot=1)
x_test,y_test = get_data_subsets(test, dur, variant, onehot=1)
inp_shape1 = (dur,N)
inp_shape2 = (x_train.shape[1]-dur,N)
x_train = [x_train[:,inp_shape2[0]:,:],x_train[:,:inp_shape2[0],:]]
x_test = [x_test[:,inp_shape2[0]:,:],x_test[:,:inp_shape2[0],:]]
"""für Dimensions-tests"""
print('x-train shape: ' + str(x_train[0].shape)+ str(x_train[1].shape))
print('y-train shape: ' + str(y_train.shape))
print('x-test shape: ' + str(x_test[0].shape)+ str(x_test[1].shape))
print('y-test shape: ' + str(y_test.shape))
print('test shape: ' + str(test.shape))
model,history = cnn_2inputs(x_train, y_train, x_test, y_test, inp_shape1, inp_shape2, epochs)
return model,history,x_train,y_train,x_test,y_test
Next Day¶
In [30]:
model_cnn_2inputs,history_cnn_2inputs,x_train,y_train,x_test,y_test = main_cnn_2inputs(returns, 100, 2000, variant='next_day', epochs=200)
print(model_cnn_2inputs.summary())
In [31]:
directional_goodness(model_cnn_2inputs, x_test, y_test)
compare_top(model_cnn_2inputs, x_test, y_test)
plots(model_cnn_2inputs, history_cnn_2inputs, x_test=x_test, x_train=x_train, y_test=y_test, y_train=y_train)
Next Month Average¶
In [32]:
model_cnn_2inputs_m,history_cnn_2inputs_m,x_train_m,y_train_m,x_test_m,y_test_m = main_cnn_2inputs(returns, 100, 2000, variant='month', epochs=200)
print(model_cnn_2inputs_m.summary())
In [33]:
directional_goodness(model_cnn_2inputs_m, x_test_m, y_test_m)
compare_top(model_cnn_2inputs_m, x_test_m, y_test_m)
plots(model_cnn_2inputs_m, history_cnn_2inputs_m, x_test=x_test_m, x_train=x_train_m, y_test=y_test_m, y_train=y_train_m)
Implied Volatility¶
In [34]:
model_cnn_2inputs_v,history_cnn_2inputs_v,x_train_v,y_train_v,x_test_v,y_test_v = main_cnn_2inputs(returns, 100, 2000, variant='vola', epochs=200)
print(model_cnn_2inputs_v.summary())
In [35]:
directional_goodness(model_cnn_2inputs_v, x_test_v, y_test_v)
compare_top(model_cnn_2inputs_v, x_test_v, y_test_v)
plots(model_cnn_2inputs_v, history_cnn_2inputs_v, x_test=x_test_v, x_train=x_train_v, y_test=y_test_v, y_train=y_train_v)
Expectations:
- CNN with additional information of industry affiliation seems to detect known patterns of stock correlations in the same category
- This should lead to better predictions
Results:
- Didn´t help considerably to enhance prediction power
- could be the result of overfitting due to industry labels
- Next Month Average overestimating low values and underestimating high values
- Volatility Input overestimating low values and underestimating low values
Conclusion for CNNs:
- Patterns in stocks seem to be to variable to be efficiently detected and do not contribute to effective prediction power in our calculation regime
- Learning rate converged
- Unfit for prediction
Next Day¶
In [38]:
model_fully,history_fully,x_train,y_train,x_test,y_test = main_fully(returns, 100, 2000, variant='next_day', epochs=200)
print(model_fully.summary())
In [39]:
directional_goodness(model_fully, x_test, y_test)
compare_top(model_fully, x_test, y_test)
plots(model_fully, history_fully, x_test=x_test, x_train=x_train, y_test=y_test, y_train=y_train)
Next Month Average¶
In [40]:
model_fully_m, history_fully_m, x_train_m,y_train_m,x_test_m,y_test_m = main_fully(returns, 100, 2000, variant='month', epochs=200)
print(model_fully_m.summary())
In [41]:
directional_goodness(model_fully_m, x_test_m, y_test_m)
compare_top(model_fully_m, x_test_m, y_test_m)
plots(model_fully_m, history_fully_m, x_test=x_test_m, x_train=x_train_m, y_test=y_test_m, y_train=y_train_m)
Implied Volatility¶
In [42]:
model_fully_v,history_fully_v,x_train_v,y_train_v,x_test_v,y_test_v = main_fully(returns, 100, 2000, variant='vola', epochs=200)
print(model_fully_v.summary())
In [43]:
directional_goodness(model_fully_v, x_test_v, y_test_v)
compare_top(model_fully_v, x_test_v, y_test_v)
plots(model_fully_v,history_fully_v, x_test=x_test_v, x_train=x_train_v, y_test=y_test_v, y_train=y_train_v)
Expectations:
- The fully connected Neural Network can fit "theoretically everything"
- limited Prediction Power since unspecialised
Results:
- trains slowly
- prediction power subpar
- error seems to have happened in next month average prediction
- model could have been optimized before input
Next Day¶
In [46]:
model_gru,history_gru,x_train,y_train,x_test,y_test = main_gru(returns, 100, 2000, variant='next_day', epochs=200)
print(model_gru.summary())
In [47]:
directional_goodness(model_gru, x_test, y_test)
compare_top(model_gru, x_test, y_test)
plots(model_gru, history_gru, x_test=x_test, x_train=x_train, y_test=y_test, y_train=y_train)
Next Month Average¶
In [48]:
model_gru_m,history_gru_m,x_train_m,y_train_m,x_test_m,y_test_m = main_gru(returns, 100, 2000, variant='month', epochs=200)
print(model_gru_m.summary())
In [49]:
directional_goodness(model_gru_m, x_test_m, y_test_m)
compare_top(model_gru_m, x_test_m, y_test_m)
plots(model_gru_m, history_gru_m, x_test=x_test_m, x_train=x_train_m, y_test=y_test_m, y_train=y_train_m)
Implied Volatility¶
In [50]:
model_gru_v,history_gru_v,x_train_v,y_train_v,x_test_v,y_test_v = main_gru(returns, 100, 2000, variant='next_day', epochs=200)
print(model_gru_v.summary())
In [51]:
directional_goodness(model_gru_v, x_test, y_test)
compare_top(model_gru_v, x_test_v, y_test_v)
plots(model_gru_v, history_gru_v, x_test=x_test_v, x_train=x_train_v, y_test=y_test_v, y_train=y_train_v)
Expectations:
- RNN fits stock market data good, specialised on time series prediction
- GRU learns faster but less effective
- Choice of hyperparameters essential
Results:
- Underfits Changes by a high margin
- Close to Moving Average
- Some spikes were recognized but learning rate almost converged
Next Day¶
In [54]:
model_gru_2inputs,history_gru_2inputs,x_train,y_train,x_test,y_test = main_gru_2inputs(returns, 100, 2000, variant='next_day', epochs=200)
print(model_gru_2inputs.summary())
In [55]:
directional_goodness(model_gru_2inputs, x_test, y_test)
compare_top(model_gru_2inputs, x_test, y_test)
plots(model_gru_2inputs, history_gru_2inputs, x_test=x_test, x_train=x_train, y_test=y_test, y_train=y_train)
Next Month Average¶
In [56]:
model_gru_2inputs_m,history_gru_2inputs_m,x_train_m,y_train_m,x_test_m,y_test_m = main_gru_2inputs(returns, 100, 2000, variant='month', epochs=200)
print(model_gru_2inputs_m.summary())
In [57]:
directional_goodness(model_gru_2inputs_m, x_test_m, y_test_m)
compare_top(model_gru_2inputs_m, x_test_m, y_test_m)
plots(model_gru_2inputs_m, history_gru_2inputs_m, x_test=x_test_m, x_train=x_train_m, y_test=y_test_m, y_train=y_train_m)
Implied Volatility¶
In [58]:
model_gru_2inputs_v,history_gru_2inputs_v,x_train_v,y_train_v,x_test_v,y_test_v = main_gru_2inputs(returns, 100, 2000, variant='vola', epochs=200)
print(model_gru_2inputs.summary())
In [59]:
directional_goodness(model_gru_2inputs_v, x_test_v, y_test_v)
compare_top(model_gru_2inputs_v, x_test_v, y_test_v)
plots(model_gru_2inputs_v, history_gru_2inputs_v, x_test=x_test_v, x_train=x_train_v, y_test=y_test_v, y_train=y_train_v)
Expectations:
- Additional Industry information adds underlying structure
- Fits become better than regular GRU RNN
Results:
- Underfits Stock Data by a high margin
- Learning curves converged very fast
- Probably misspecified model
Rule of thumb for Number of Hidden Nodes¶
$$n_h \approx {2(n_i + n_o) \over 3}$$$n_i$ number of stocks
$n_o$ number of days
Next Day¶
In [62]:
model_lstm,hisory_lstm,x_train,y_train,x_test,y_test = main_lstm(returns, 100, 2000, variant='next_day', epochs=200)
print(model_lstm.summary())
In [63]:
directional_goodness(model_lstm, x_test, y_test)
compare_top(model_lstm, x_test, y_test)
plots(model_lstm, hisory_lstm,x_test=x_test, x_train=x_train, y_test=y_test, y_train=y_train)
Next Month Average¶
In [64]:
model_lstm_m,hisory_lstm_m,x_train_m,y_train_m,x_test_m,y_test_m = main_lstm(returns, 100, 2000, variant='month', epochs=200)
print(model_lstm_m.summary())
In [65]:
directional_goodness(model_lstm_m, x_test_m, y_test_m)
compare_top(model_lstm_m, x_test_m, y_test_m)
plots(model_lstm_m, hisory_lstm_m, x_test=x_test_m, x_train=x_train_m, y_test=y_test_m, y_train=y_train_m)
Implied Volatility¶
In [66]:
model_lstm_v,hisory_lstm_v,x_train_v,y_train_v,x_test_v,y_test_v = main_lstm(returns, 100, 2000, variant='vola', epochs=200)
print(model_lstm_v.summary())
In [67]:
directional_goodness(model_lstm_v, x_test_v, y_test_v)
compare_top(model_lstm_v, x_test_v, y_test_v)
plots(model_lstm_v, hisory_lstm_v,x_test=x_test_v, x_train=x_train_v, y_test=y_test_v, y_train=y_train_v)
Expectations:
- Long-Short Term Memory RNN will learn slower than regular RNN
- Should learn underlying time-series structure better than GRU RNN
Results:
- Like GRU does underestimate return values
- Learning rate plots show that not all input spaces show the same rate of convergence
- Longer Training times may further improve results
Next Day¶
In [70]:
model_lstm_2inputs,history_lstm_2inputs,x_train,y_train,x_test,y_test = main_lstm_2inputs(returns, 100, 2000, variant='next_day', epochs=200)
print(model_lstm_2inputs.summary())
In [71]:
directional_goodness(model_lstm_2inputs, x_test, y_test)
compare_top(model_lstm_2inputs, x_test, y_test)
plots(model_lstm_2inputs, history_lstm_2inputs, x_test=x_test, x_train=x_train, y_test=y_test, y_train=y_train)
Next Month Average¶
In [72]:
model_lstm_2inputs_m,history_lstm_2inputs_m,x_train_m,y_train_m,x_test_m,y_test_m = main_lstm_2inputs(returns, 100, 2000, variant='next_day', epochs=200)
print(model_lstm_2inputs_m.summary())
In [73]:
directional_goodness(model_lstm_2inputs_m, x_test_m, y_test_m)
compare_top(model_lstm_2inputs_m, x_test_m, y_test_m)
plots(model_lstm_2inputs_m, history_lstm_2inputs_m, x_test=x_test_m, x_train=x_train_m, y_test=y_test_m, y_train=y_train_m)
Implied Volatility¶
In [74]:
model_lstm_2inputs_v,history_lstm_2inputs_v,x_train_v,y_train_v,x_test_v,y_test_v = main_lstm_2inputs(returns, 100, 2000, variant='next_day', epochs=200)
print(model_lstm_2inputs_v.summary())
In [75]:
directional_goodness(model_lstm_2inputs_v, x_test_v, y_test_v)
compare_top(model_lstm_2inputs_v, x_test_v, y_test_v)
plots(model_lstm_2inputs_v, history_lstm_2inputs_v, x_test=x_test_v, x_train=x_train_v, y_test=y_test_v, y_train=y_train_v)
Expectations:¶
- LSTM with OHE additional Information should learn best due to (also) most complete state of information and best suited type for time series prediction
Results:¶
- Learning curves also converged very fast implying either to have found a correlation right from the beginning or a misspecification of network model when concatenating the networks
- also unfit for prediction
Comparison between LSTM and GRU:¶
- GRU with two gates:
- reset gate
- update gate
- LSTM with three gates:
- input gate
- forget gate
- output gate
- Update gate in GRU $\approx$ input & forget gate in LSTM
- GRU needs less parameters
- GRU can better generalize
- GRU always exposes all the memory state, LSTM learns more fine distinctions due to controlled variable memory state
Discussion and Outlook¶
Neural Networks don´t really do a good job on predicting stock markets
Patterns are too variable to be recognised as patterns by machines
time series prediction works ok
Input Data Transformations can be optimized, so that maybe structure is recognizable
- smoothing, space transformation, clustering, etc
Neural Networks can be optimized
- hyperparameters are of the essence with most NNs
- input shape (time window, stock number)
- number of different kinds of neurons, layers, etc.
- activation functions
More computational power could help
- sometimes NNs have a "breakthrough" in learning curve after what seems like convergence
One-Hot Encoded Information of Industry sectors usually improves prediction power (by a small margin)
- Firms Evidence that Underlying Industry Sector Information is relevant
large scale study of these comparisons would still be interesting since it seems like not all networks have learned to convergence
Learning curves of networks with OHE additional Information show weird behaviour where further investigation into network model building is promising
In [ ]: